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ABSTRACT 

Objective To explore the relation between study 
concordance, take home message, funding, and 
dissemination of comparative studies assessing the 
effects of influenza vaccines. 
Design Systematic review without meta-analysis. 
Data extraction Search of the Cochrane Library, PubMed, 
Embase, and the web, without language restriction, for 
any studies comparing the effects of influenza vaccines 
against placebo or no Intervention. Abstraction and 
assessment of quality of methods were carried out. 
Data synthesis We identified 259 primary studies (274 
datasets). Higher quality studies were significantly more 
likely to show concordance between data presented and 
conclusions (odds ratio 16.35, 95% confidence interval 
4.24 to 63.04) and less likely to favour effectiveness of 
vaccines (0.04, 0.02 to 0.09). Governmentfunded studies 
were less likely to have conclusions favouring the vaccines 
(0.45, 0.26 to 0.90). A higher mean journal impact factor 
was associated with complete or partial industry funding 
compared with government or private funding and no 
funding (differences between means 5.04). Study size was 
not associated with concordance, content of take home 
message, funding, and study quality. Higher citation index 
factor was associated with partial or complete industry 
funding. This was sensitive to the exclusion from the 
analysis of studies with undeclared funding. 
Conclusion Publication in prestigious journals is 
associated with partial or total industry funding, and this 
association is not explained by study quality or size. 

INTRODUCTION 

Healthcare workers wanting to keep up to date with 
recent advances in their specialty must deal with the 
quantity and quality of information sources. 12 One 
study estimated that every month journals publish 
7287 items (studies, letters, and editorials) relevant to 
primary care. These would take physicians trained in 
epidemiology 627 hours to read and appraise. Few 
healthcare workers have the time and skills to carry out 
in depth critical appraisal of published articles. 3 Given 
the pressures of everyday work, the time available for 



reading a scientific article might be as little as 
22 minutes. 4 

To inform their conduct and update their knowledge 
physicians might rely on a brief (two minute) scan of 
material. 56 This probably includes either browsing the 
abstract (the "shop window" of an article) or the 
conclusions paragraph (the "take home message"), 
especially those published in prestigious journals (that 
is, those with the highest journal impact factor) that are 
most readily available or of their digests. The impact 
factor was specifically developed to facilitate prioritisa- 
tion of subscription resources, allowing targeting on the 
most cited journals, considered to be most read and the 
ones that publish articles of the best quality. These 
journals are thus more likely to be accessible and their 
content widely disseminated. 78 

Methodological quality of studies is the other impor- 
tant aspect. Several items, all affecting the reliability of 
what presents to healthcare workers, are related in a 
complex fashion. Such quality is associated with the size 
of the effects reported in a study. The lower the quality, 
the greater the effect of the intervention seems. 910 

Funding of meta-analyses of antihypertensive drugs 
by individual pharmaceutical companies has been 
shown to be inversely associated with whether data 
presented supported the authors' conclusions 
(concordance) 11 and directly associated with the 
probability that study conclusions did or did not 
support the drugs being evaluated (take home 
message). 12 The relation between the items of quality, 
concordance, take home message, and funding of 
Single studies, however, is thought to be specific to the 
context, drugs evaluated, and outcomes assessed. This 
implies that generalisation of quality across study 
designs, interventions, and outcomes cannot be made 
on current evidence. In addition we found no studies 
looking at the relation of all these variables with that of 
study dissemination and possible impact. 13 14 

We explored the relation between study concor- 
dance, take home message, funding, and citation (as a 
proxy for dissemination) in studies on influenza 
vaccines by reviewing all published comparative 
studies. Our choice was motivated by the global nature 
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of the Intervention, the almost exclusive analysis by 
analytical studies of quality items in randomised 
controlled trials or their meta-analyses (in influenza 
vaccines there is an abundance of non-randomised 
evidence), and by the recent publication of studies 
suggesting poor quality and contradictory evidence 
from field studies supporting vaccination. 15 20 

METHODS 

Evidence searches 

We systematically reviewed comparative studies on 
influenza vaccines. We used the bibliography of 
available up to date systematic reviews of influenza 
vaccines to identify studies comparing the effects of 
influenza vaccines with placebo or no Intervention 
("primary studies") and carried out supplementary 
searches for all other comparative primary studies 
published by the end of 2006 (see appendix on 
bmj.com for search strategy and procedure). 

Data extraction and quality assessment 

We extracted primary study data in duplicate accord- 
ing to a list of variables (box 1). 

Two reviewers, independently unblinded to the 
authors and institutions of the article, extracted data 
from the studies in three phases. Firstly, they read each 
study and compüed an abstract describing study content. 
Data presented in the study were summarised on a 
computerised database based on the Cochrane vaccines 
field study register. This process fnvolved understanding 
the design and intent of the authors as well as reading and 
reconciling each part of the study. We then compared 
our abstract with the authors' presentation and 

Box 1 Variables extracted from primary studies 

Study identification 

Year(s) of execution of the study (the end of the past influenza season cited in the study) 
Yearof publication 
Study design and location 
Numberof arms 

Comparison and type of outcome assessed within the same study (papers divided into 

separate studies or datasets if they contained more than one study or comparison) 

Study population 

Types of influenza Vaccine used 

Content of influenza vaccines used 

Match between content and circulating viruses in the Community 
Types of outcome assessed 
Intervention arm size 

Numberof events observed in the Intervention arms 
Types of control 
Size of control arms 

Numberof events observed in the control arms 
Type of funder 

Impact factorassigned to the journal fortheyearafterthe study was published, numberof 
times the articles have been cited by other sources (citation index factor), conclusions of 
article (classified according to the categories reported in box 2) 
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Bibliography of systematic reviews on influenza Vaccine and 
bibliographic supplementary searches identified 12 000 citations 

~T 

317 papers (338 comparative studies 
datasets) considered for inclusion 

24 studies/datasets excluded forvarious reasons 
— »- 40 comparative studies/datasets excluded because 
they were head to head 

259 papers (274 comparative studies/datasets) included 
in review (5 papers contain two comparative studies each) 

~T 

RCT: 109 papers (1 1 6 datasets) 
CCT: 22 papers (22 datasets) 
Prospective cohorts: 67 papers (69 datasets) 
Retrospective cohorts: 46 papers (47 datasets) 
Case-control studies: 20 papers and datasets 



Flow of primary studies in review (RCT=randomised controlled 
trial; CCT=clinical controlled trial, semi-randomised trial) 

summaries of results. To classify concordance and take 
home message definitions we asked whether data 
presented in the study supported the authors' conclu- 
sions and then whether the conclusions did or did not 
support the vaccine(s) being evaluated. We classified 
take home message of all studies as supporting, neutral, 
or critical of the effects of influenza vaccines (box 2) . The 
definitions were based on work carried out on study 
concordance and take home message. 11 12 We grouped 
funding sources into govemment/private/unfumded or 
industry/rnixed, concordance into yes or no/partialfy/ 
unclear, and take home message into favourable or 
rnixed/ unfavourable/ unclear. 

Finally, we assessed quality of randomised and non- 
randomised studies separately. Randomised studies 
were assessed according to randomisation method, 
generation of the allocation sequence, allocation 
concealment, blinding, and follow-up. Non-rando- 
mised studies were assessed for the presence of 
potential confounders with the appropriate Newcas- 
tle-Ottawa scales 29 for case-control and cohort studies. 
We assigned risk of bias (low, moderate, and high risk 
of bias) as described by the Cochrane Handbook 30 by 
design category (randomised and non-randomised). 

Statistical methods 

We used x 2 tests to assess the association between items 
such as quality of methods, funding sources, coher- 
ence, and take home message. 

To assess the level of dissemination we identified the 
impact factor of the journals publishing each of the 
studies in the review. A journal's impact factor is 
calculated from the number of citations in the current 
year to items published in the previous two years 
(numerator), divided by the number of Substantive 
articles and reviews published in the same two years 
(the denominator). This is proposed as a proxy for 
journal quality and, by implication, for quality of the 
research articles in that journal. 78 The impact factor 
was designed to identify journals worthy of 
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subscription, and its rankings fit well with the perceived 
prestige of joumals 8 and their circulation. 

We identified the citation index of each article (the 
cumulative total of the number of times the article has 
been cited in joumals indexed in the ISI Web of 
Science database) . The latest impact factor and citation 
indices were taken from the ISI Website http:// 
isiwebofknowledge.com/. 



Box 2 Classification of study take home message 
Supportive 

• The authors State the dominance of the vaccines even without supporting Statistical 
conclusions 

• The study reports statistically significant Vaccine effects compared with control 
intervention(s) 

• The authors report the main study end points as statistically significantly dominant 
(P<0.05) compared with the control arm(s) 

• The authors State thatvaccination reducesdisease bürden and/ortreatmentcostsand/ 
or resource consumption 

• The authors stress the efficacy, effectiveness, and safety of the vaccines compared with 
control intervention(s) 

• The authors conclude that vaccination should be undertaken 

• The authors criticise dissenters on the effects of vaccination 

• The authors support at least one of the producers' or Programme Sponsors' Statements 
on the vaccines 

Neutral 

• The authors conclude that there are insufficient data to assess one or all of the effects of 
the vaccines 

• The authors make no recommendations but seem fairly to weigh up pros and cons of 
each course of action 

• The authors conclude that vaccination is cost neutral 

• The authors conclude that the data presented are insufficient to answer the study 
question 

• The authors State that after vaccination the evidence for reduction of disease bürden 
and/ortreatment costs and/or resource consumption is slight or non-existent 

• The authors dissent from at least one of the producers' or Programme Sponsors' 
Statements on the vaccines but do not reach a definite conclusion 

• The authors conclude that the evidence shows efficacy, effectiveness, and safety of the 
vaccines compared with control intervention(s) for some outcomes but not for others 

• The authors conclude that the evidence shows efficacy, effectiveness, and safety of one 
type of Vaccine compared with anotherand with control intervention(s) 

Critical/not supportive 

• The authors report no statistically significant efficacy or effectiveness results and/or 
significant harms 

• The authors clearly State the vaccines' inferiority and present data showing significant 
Vaccine inferiority compared with control(s) 

• The authors State thatvaccination does not reduce disease bürden and/ortreatment 
costs and/or resource consumption 

• The authors emphasise concerns over efficacy and/or effectiveness and/or safety and/ 
or recommend alternative interventions to control the bürden of disease 

• The authors criticise those supporting influenza vaccination or State that vaccination 
should not be introduced or continued 

• The authors are critical of all the producers' or Programme Sponsors' Statements on the 
vaccines 



We also tested the association between delay in 
publication (difference between the publication date 
and the date the study was completed), the journal 
impact factor, or the citation index factor against the 
items described above by Wilcoxon and Kruskal- 
Wallis non-parametric tests (SAS/STAT 9.1, SAS, 
Cary, NC) . Other studies have described an association 
between type of take home message and citation and 
type of funder and citation. 1221 " 28 

RESULTS 

Characteristics of primary studies 

The figure shows the flow of the included studies. 

We included 274 comparative studies/datasets (from 
259 papers) in our population. These were mostly 
randomised or semi-randomised (that is, the methods of 
allocation were alternation or birth date) studies (138/ 
274, 50%) or prospective cohort studies (69/274, 25%). 
Most studies were carried out on healthy or general 
populations (203/274, 74%) and were government 
financed (48%). Seventy per cent of studies reported 
conclusions favourable to the vaccines but only 18% 
showed complete concordance between data reported 
and study conclusions. Over half (56%) of studies were at 
high risk of bias, with only 4% being at low risk. We did 
not include any Vaccine registration trials. 

Association between study quality and concordance, take 
home message, and funding 

Table 1 summarises our main findings. All compar- 
isons assessing the association with study quality were 
carried out after we excluded studies for which we had 
insufficient data (4/274). We calculated % 2 on the basis 
of the three methodological quality categories of high, 
moderate, and low risk of bias. We calculated the % 2 for 
all comparisons of funding sources for three categories 
(government, industry, missing). Because a high 
proportion of studies had missing funding information 
(64/274, 23%) we carried out a füll sensitivity analysis 
to assess robustness of observed associations by 
systematically assigning studies with unclear funding 
source to the other known funding categories, thus 
covering all possible scenarios. 

Our analysis shows the presence of a strong positive 
association between methodological quality and con- 
cordance between results presented and study conclu- 
sions (when we aggregated studies with moderate and 
high risk of bias: odds ratio 16.35, 95% confidence 
interval 4.24 to 63.04). In other words, the higher the 
study quality (and the lower the risk of bias), the higher 
the probability of concordance. In addition, the higher 
the probability of concordance, the lower the prob- 
ability that a study's conclusions were in favour of 
vaccines' effectiveness (0.04, 0.02 to 0.09); table 2 
shows examples of discordance between results and 
conclusions and take home message. We found no 
association between funding source and methodologi- 
cal quality (excluding studies with unknown funding: 
0.74, 0.19 to 2.84), but there was aninverse association 
between conclusions in favour of the vaccines' effec- 
tiveness and government funding (0.45, 0.26 to 0.90). 
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Table 1 1 Summary of main findings of assessment of possible association between methodological quality, concordance between data presented and 
conclusions reported, take home message of article conclusions, and funding source in comparative studies of influenza vaccines 


Is there a relation between: 


Odds ratio (95% Cl) 


Interpretation 


Sensitivity analysis carried out? If yes what were the results? 


Methodological quality and concordance 
between data presented and 
conclusions reported? 


Aggregate moderate and high risk of 
bias: 16.35 (4.24 to 63.04) 


Positive association between low risk of 
bias and concordance 


No 


Methodological quality and funding 
source? 


Aggregate moderate and high risk of 
bias, excluding studies with missing 
funding source: 0.74 (0.19 to 2.84) 


No evidence of negative association 
between governmental funding source 
and low risk of bias 


Yes. Sensitivity analysis carried out on 240 possible scenarios: 
1.64%ofscenarioswithOR<l significant, 65.4%withOR<l non- 
significant; 0.83% with OR=l; and 32.1% with OR >1 non- 
significant at 5% level 


Methodological quality and take home 
message? 


Aggregate moderate and high risk of 
bias: 0.19 (0.05 to 0.64) 


Negative association between low risk of 
bias and favourable conclusion 


No 


Concordance between data presented 
and conclusions reported and take home 
message? 


0.04 (0.02 to 0.09) 


Negative association between presence 
of concordance and favourable 
conclusion 




Concordance between data presented 
and conclusions reported and funding 
source? 


Excluding studies with missing 
fundingsource: 1.47 (0.72 to 3.07) 


No evidence of positive association 
between concordance and government 
funding source at 5% significance level 


Yes. Sensitivity analysis carried out on 413 possible scenarios; 
16.5% of scenarios with 0R<1 and non-significant, 57.9% with 
OR >1 non-significant, and 25.7% with OR >1 significant at 5% 
level 


Fundingsource and take home message? 


Excluding studies with missing 
funding source: 0.45 (0.26 to 0.90) 


Evidence of negative association between 
favourable conclusion and government 
funding 


Yes. Sensitivity analysis carried out on 989 possible scenarios; 
47.0% of scenarios with 0R<1 significant, 38.5% with 0R<1 non- 
significant, 14.4% with OR >1 non-significant, and only 0.1% 
(one scenario) with OR >1 significant at 5% level 



This finding confirms the established association 
between funding source and type of study 
conclusions. 21 " 2528 

Association between study quality items, funding, and 
dissemination 

Statistical tests show no evidence of association 
between the mean time lapsed from the end of the 
study to the date of publication (publication delay), 



methodological quality, type of study design, concor- 
dance, coherence, and funder. There was no evidence 
of association between either journal impact factor or 
citation index factor and study design, methodological 
quality, and concordance. 

The mean journal impact factor of 92 government 
funded studies was 3.74, and their mean citation index 
factor was 33.75. We calculated the mean impact factor 
on 1 14 studies, as the oldest studies were published in 



Table 2 1 Examples of discrepancy between results and conclusions 



Study 

Wongsurakiat 2004 31 
(randomised placebo 
controlled trial in 125 
adults and older people 
with COPD) 



Wilde 1999" 
(randomised trials on 
264 healthy healthcare 
workers, three 
consecutive seasons) 



Summary of results 

Significant reduction shown for total acute respiratory infections (ARI) with 
laboratory confirmation of influenza (RR 0.24, 95% Cl 0.06 to 0.7; P=0.005) 
and influenza like illness (ILI) (0.34, 0.1 to 0.99, P=0.03). For acute 
exacerbations, difference not significant (0.92, 0.67 to 1.3, P=0.6). In this 
category 13/21 confirmed cases of influenza isolated, so, as for total ARI 
episodes (124 in vaccinated and 145 in placebo group), there was no 
difference between two intervention groups. This applies also to "probability of 
not being admitted to hospital related to ARI (P=0.2 by log ranktest) and 
probability of not receiving mechanical Ventilation related to ARI (P=0.4 by log 
ranktest)" 

Influenza infection (fourfold increase in haemagglutination inhibitingantibody 
against virus A or B between serum sample after immunisation and after 
epidemic). Efficacy against A (H3N2) virus estimated as 88% (47% to 97%, 
P=0.001); against B virus as 89% (14% to 99%, P=0.02). Authors' conclusions 
about efficacy derived from cumulative data only. They apply effect measure 
and significance test (x 2 ) to cumulative data only. When applied to Single 
comparisons, significance reached only for influenza A in 1992-3 (P=0.026). 
Days with respiratory illness (52 v73 days; P=0.57; mean 0.29 (SD 0.68) days v 
0.41 (SD 1.0) days, and absence due to illness (18 v38 days; P=0.41, mean 0.1 
(SD 0.35) days 1/0.21 (SD 0.75) days nodifferent amongvaccinated and control 
group (three seasons' cumulative data) 



Author's conclusions and reviewer comments 

Study was conducted over one year. Conclusions support recommendation of 
annual vaccination (one dose is sufficient in adults, as strong response has been 
observed). Authors note that Vaccine effectiveness has been shown, even if it was 
possibly administered too late (in region where study was carried out peak 
incidence of influenza occurs usually in May). Comment: though authors State that 
effectiveness is shown for influenza related ARI only, and not influenza, they 
recommend vaccination for patients with COPD. This means recommendingvaccine 
though it is not effective against influenza and acute exacerbations. In addition, lack 
of comment on Community viral circulation and Vaccine content and matchingmake 
verification of effectiveness against ARI impossible 

"In conclusion, influenza Vaccine is effective in preventingserologically proven 
influenza infection in young, healthy hospital-based healthcare Professionals and 
may reduce cumulative days of illness and absence. These data suggest that a 
policy of annual immunization with influenza Vaccine in healthcare Professionals 
will reduce influenza infections and can reduce associated illness." Comment: 
influenza vaccination is recommended though outcomes are exclusively serological 
(Surrogates) calculated in aggregate overthreeyears (180 in intervention arms and 
179 in three different control arms). Clinical outcomes are not significantly affected 
by vaccination 



Carman 2000 33 (block 
randomised trial, 20 long 
term care hospitals) 



All cause mortality less common in long term care where vaccination was 
offered (vaccination rate 13.6%; 102/749) than in those where it was not 
(vaccination rate 22.4%; 154/688). Crude OR 0.58, 0.40 to 0.84, P=0.014). 
Significance disappears after adjustment for degree of disability by Barthel 
scale, age, sex, vaccination of patients (0.61, 0.36 to 1.04, P=0.092). 
Virologicalsurveillance (routine, from some symptomatic subjects, from some 
samples taken at death) did not show different frequency of viruses A or B 
isolates between two groups (culture and PCR) 



"Vaccination of health-care workers was associated with a substantial decrease in 
mortality [for all causes] among patients. However, virological surveillance showed 
no associated decrease in non-fatal influenza infection in patients." Comment: 
implausible conclusion with use of all cause mortality an outcome lacking 
specificity. tong list of confounders: biased reporting of autopsy sampling, 
trenchant conclusion despite apparent lack of effect on viral circulation, brief 
description of Vaccine content or matching (in discussion), attrition in serology 
follow-up, possible selection bias of healthcare workers and patients, higher 
Barthel score in vaccinated arm. Once data were adjusted for Barthel score, age, and 
sex no effect was observed 



RCT=randomised controlled trial, COPD=chronic obstructive pulmonary disease, RR=relative risk, OR=odds ratio, PCR=polymerase chain reaction. 
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joumals for which no impact factor was available. For 
the 52 studies wholly or partly funded by industry, the 
mean impact factor was 8.78 and the mean citation 
index factor was 58.39. We calculated the mean 
citation index for 74 studies as the oldest studies were 
published in journals for which no citation index was 
available. 

Non-parametric testing shows that the impact factor 
score of studies with complete or partial industry 



funding was significantly higher than that of studies 
funded by government and other funders (table 3). 

Non-parametric testing for article citation index 
showed a similar significant difference, which was, 
however, sensitive to the exclusion of studies with no 
declared funding source. In this case we found no 
difference in citation between studies with complete or 
partial funding from industry and studies funded by 
government and other funders (table 3). 



Table 3 1 Analysis of relation between journal impact factor, citation index factor, study sample size, publication delay, methodological quality, take home 
message content, concordance, and source of funding 



Is there a relation between: 


Statistical tests* 


Interpretation 


Sensitivity 
analysis carried 
out? 


If yes, what were results? 


Methodological quality and JIF (with 
aggregate moderate and high risk of 


z=1.3, P=0.184 


No evidence of difference in mean JIF between studies with low risk of 
bias and studies with (high or moderate) risk of bias 


No 




Methodological quality and CIF (with 
aggregate moderate and high risk of 


z=-0.19, P=0.851 


No evidence of difference in mean CIF between studies with low risk of 
bias and studies with (high or moderate) risk of bias 


No 


- 


Methodological quality and CSS 
(with aggregate moderate and high 


z=-0.96, P=0.338 


No evidence of difference in mean CSS between studies with low risk of 
bias and studies with (high or moderate) risk of bias 


Yes (only with RCT) 


No change in interpretation 


Methodological quality and PD (with 
aggregate moderate and high risk of 


z=-0.38, P=0.707 


No evidence of difference in mean PD between studies with low risk of 
bias and studies with (high or moderate) risk of bias 


No 


- 


Take home message and |IF? 


z=-1.51, P=0.131 


No evidence of difference in mean JIF between studies with favourable 
take home message and studies with mixed/unfavourable take home 


No 


- 


Take home message and CIF? 


z=-1.84, P=0.065 


No evidence of difference in mean CIF between studies with favourable 
take home message and studies with mixed/unfavourable take home 


No 


- 


Take home message and CSS? 


z=-0.41, P=0.682 


No evidence of difference in mean CSS between studies with favourable 
take home message and studies with mixed/unfavourable take home 
message 


Yes (only with RCT) 


No change in interpretation 


Take home message and PD? 


z=-0.89 P=0.375 


No evidence of difference in mean PD between studies with favourable 
take home message and studies with mixed/unfavourable take home 
message 


No 




Concordance between data 
presented and conclusions reported 
and JIF? 


z=l.l, P=0.273 


No evidence of difference in mean JIF between studies with concordance 
(yes) and studies with concordance (no/part/unclear) 


No 




Concordance between data 
presented and conclusions reported 
and CIF? 


z=0.35 P=0.729 


No evidence of difference in mean CIFbetween studies with concordance 
(yes) and studies with concordance (no/part/unclear) 


No 




Concordance between data 
presented and conclusions reported 
and CSS? 


z=0.84, P=0.404 


No evidence of difference in mean CSS between studies with 
concordance (yes) and studies with concordance (no/part/unclear) 


Yes (only with RCT) 


No change in interpretation 


Concordance between data 
presented and conclusions reported 
and PD? 


z=-0.58, P=0.563 


No evidence of difference in mean PD between studies with concordance 
(yes) and studies with concordance (no/part/unclear) 






Funding source and JIF? 


X 2 =27.4, df=2, P<0.001 


Evidence of difference in mean JIF between studies with industry funding 
sourceand otherfundingsource. Mean JIFsignificantlygreaterin industry 
funded studies than studies with other funding source 


Excluding studies 
with undeclared 
funding 


No change in interpretation 


Funding source and CIF? 


X 2 =13.5, df=2, P<0.001 


Evidence of difference in mean CIFbetween studies with industry funding 
source and otherfunding source. Mean CIF significantly greater in 
industry funded studies than studies with other funding source 


Excluding studies 
with undeclared 
funding 


Change in interpretation: "no 
evidence" 


Funding source and CSS? 


X 2 =0.06, df=2, P=0.997 


No evidence of difference in mean CSS between industry funded studies 
and government funded studies 


Excluding studies 
with undeclared 
funding and only 
with RCT 


No change in interpretation 


Funding source and PD? 


X 2 =0.97, df=2, P=0.616 


No evidence of difference in mean PD between industry funded studies 
and government funded studies 


Excluding studies 
with undeclared 
funding 


No change in interpretation 


JIF=journal impact factor; CIF=citation 
study). 

*Kruskal Wallis (x 2 ) or Wilcoxon (z). 


index factor; RCT=randomised controlled trial; CSS=comparator sample size; PD=publication delay (difference betweer 


i publication year and end of 



BMJ | ONUNE FIRST | bmj.com 



page 5 of 7 



RESEARCH 



WHAT IS ALREADY KNOWN PN THIS TOPIC 

Study sponsorship is associated with optimistic results 

Influenza vaccination continues to be recommended globally, despite growing doubts about 
the validity of the scientific evidence underpinning policy recommendations 

WHAT THIS STUDY ADDS 

Evidence is of poor quaiity, and studies with conclusions in favourof vaccines are of 
significantly lower methodological quaiity 

Influenza vaccines studies sponsored by industry are published in journals with higher impact 
factors and are cited more but are of similarsize and quaiity to the others 



There was no evidence of association between 
sample size of comparator and study design, metho- 
dological quaiity, concordance, and take home mes- 
sage. 

We conclude that study size is an unlikely explana- 
tion for the association between industry sponsorship 
and publication in journals with higher impact factors 
and that industry funding is associated with a higher 
probability of publication in journals with a higher 
impact factor and possibly a higher citation index. 

The differences in impact factor means between 
studies funded by government and those funded by 
industry or mixed Sponsors were significant and 
sufficiently robust. Whole or part industrial funding 
was associated with publication in journals with higher 
impact factors. 

DISCUSSION 

In the studies we included, poor methodological 
quaiity was associated with a discrepancy between 
results and conclusions, and this in turn was associated 
with optimistic conclusions in non-government spon- 
sored studies. We found no direct association between 
these findings and industry funding, but this might have 
been affected by the sizeable number of studies with 
undeclared Sponsors (23%). 

Studies partly or completely sponsored by industry, 
however, were published in more prestigious journals 
and are probably cited more, although their methodo- 
logical quaiity and size were similar. Some of these 
findings might help to explain the continuation of a 
near global policy, despite growing doubts as to its 
scientific basis. 

We reasoned that the combined impact factor and 
citation index would give us a good idea of the 
circulation and dissemination of the study (that is, the 
interest its publication generated). Most of our studies 
(70%) were of poor quaiity with overoptimistic 
conclusions — that is, not supported by the data 
presented. Those sponsored by industry had greater 
visibility as they were more likely to be published by 
high impact factor journals and were likely to be given 
higher prominence by the international scientific and 
lay media, despite their apparent equivalent methodo- 
logical quaiity and size compared with studies with 
other funders. Although differences in citation index 
by study funding are sensitive to the inclusion of the 

page 6 of 7 



large number of studies with undeclared sources of 
funding, the higher impact factor and citation index are 
probably a reflection of a higher profile of industry 
sponsored studies and a more thorough dissemination 
of their content. 

There are two possible mechanisms involved. 
Firstly, the same studies are cited more than the others, 
possibly because of the systematic nature of the 
dissemination of their results by industry. We have 
personally observed this on three recent occasions in 
which industry representatives presented abstracts or 
reprints of industry sponsored influenza Vaccine 
studies to decision makers, their advisors, and local 
researchers in an effort to influence their decisions. 
Symposiums, Conferences, and other types of publica- 
tion further enhance the dissemination process. Often 
the abstracts were expensively bound and translated 
into the local language, a tangible sign of their 
importance to industry. 

We cannot say for certain why industry sponsored 
studies are more attractive to more prestigious journals, 
but such journals are preferentially targeted by all 
studies because of their prominence and prestige, so 
industry sponsored studies might have a higher 
probability of acceptance. The two mechanisms 
might be linked, but further research, especially in 
other specialties, is required. As a measure of 
transparency for readers and authors, however, we 
recommend that once a year editors and publishers 
should post all sources of income related to the mnning 
of the journal. 

Our finding of lack of concordance between results 
and conclusions is similar to those of Yank and 
colleagues in an industry sponsored meta-analysis of 
antihypertensive drugs. 11 In our studies, however, we 
found a lack of concordance between results and 
conclusions associated with poor quaiity rather than 
any specific sponsorship. Given our findings of lack of 
concordance in primary studies, the content of current 
policy might reflect the gap between results and 
conclusions — that is, synthesis of evidence for policy 
making might be carried out at two independent levels : 
that of results and that of conclusions. In most cases, 
what you see is not necessarily what you get. 
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